Fairness-Aware Client Selection for Federated Learning

Federated learning (FL) has enabled multiple data owners (a.k.a. FL clients) to train machine learning models collaboratively without revealing private data. Since the FL server can only engage a limited number of clients in each training round, FL client selection has become an important research problem. Existing approaches generally focus on either enhancing FL model performance or enhancing the fair treatment of FL clients. The problem of balancing performance and fairness considerations when selecting FL clients remains open. To address this problem, we propose the Fairness-aware Federated Client Selection (FairFedCS) approach. Based on Lyapunov optimization, it dynamically adjusts FL clients' selection probabilities by jointly considering their reputations, times of participation in FL tasks and contributions to the resulting model performance. By not using threshold-based reputation filtering, it provides FL clients with opportunities to redeem their reputations after a perceived poor performance, thereby further enhancing fair client treatment. Extensive experiments based on real-world multimedia datasets show that FairFedCS achieves 19.6% higher fairness and 0.73% higher test accuracy on average than the best-performing state-of-the-art approach.Comment: Accepted by ICME 202

Federated Learning in Big Model Era: Domain-Specific Multimodal Large Models

Multimodal data, which can comprehensively perceive and recognize the physical world, has become an essential path towards general artificial intelligence. However, multimodal large models trained on public datasets often underperform in specific industrial domains. This paper proposes a multimodal federated learning framework that enables multiple enterprises to utilize private domain data to collaboratively train large models for vertical domains, achieving intelligent services across scenarios. The authors discuss in-depth the strategic transformation of federated learning in terms of intelligence foundation and objectives in the era of big model, as well as the new challenges faced in heterogeneous data, model aggregation, performance and cost trade-off, data privacy, and incentive mechanism. The paper elaborates a case study of leading enterprises contributing multimodal data and expert knowledge to city safety operation management , including distributed deployment and efficient coordination of the federated learning platform, technical innovations on data quality improvement based on large model capabilities and efficient joint fine-tuning approaches. Preliminary experiments show that enterprises can enhance and accumulate intelligent capabilities through multimodal model federated learning, thereby jointly creating an smart city model that provides high-quality intelligent services covering energy infrastructure safety, residential community security, and urban operation management. The established federated learning cooperation ecosystem is expected to further aggregate industry, academia, and research resources, realize large models in multiple vertical domains, and promote the large-scale industrial application of artificial intelligence and cutting-edge research on multimodal federated learning

Predicting radiologists' gaze with computational saliency models in mammogram reading

Previous studies have shown that there is a strong correlation between radiologists' diagnoses and their gaze when reading medical images. The extent to which gaze is attracted by content in a visual scene can be characterised as visual saliency. There is a potential for the use of visual saliency in computer-aided diagnosis in radiology. However, little is known about what methods are effective for diagnostic images, and how these methods could be adapted to address specific applications in diagnostic imaging. In this study, we investigate 20 state-of-the-art saliency models including 10 traditional models and 10 deep learning-based models in predicting radiologists' visual attention while reading 196 mammograms. We found that deep learning-based models represent the most effective type of methods for predicting radiologists' gaze in mammogram reading; and that the performance of these saliency models can be significantly improved by transfer learning. In particular, an enhanced model can be achieved by pre-training the model on a large-scale natural image saliency dataset and then fine-tuning it on the target medical image dataset. In addition, based on a systematic selection of backbone networks and network architectures, we proposed a parallel multi-stream encoded model which outperforms the state-of-the-art approaches for predicting saliency of mammograms

Efficient Spatial-Temporal Rebalancing of Shareable Bikes (Student Abstract)

Bike sharing systems are popular worldwide now. However, these systems are facing a problem - rebalancing of shareable bikes among different docking stations. To address this challenge, we propose an approach for the spatial-temporal rebalancing of shareable bikes which allows domain experts to optimize the rebalancing operation with their knowledge and preferences without relying on learning by trial-and-error

Ethically Aligned Mobilization of Community Effort to Reposition Shared Bikes

We consider the problem of mobilizing community effort to reposition indiscriminantly parked shared bikes in urban environments through crowdsourcing. We propose an ethically aligned incentive optimization approach WSLS which maximizes the rate of success for bike repositioning while minimizing cost and prioritizing users’ wellbeing. Realistic simulations based on a dataset from Singapore demonstrate that WSLS significantly outperforms existing approaches

CAreFL: enhancing smart healthcare with contribution-aware federated learning

Artificial intelligence (AI) is a promising technology to transform the healthcare industry. Due to the highly sensitive nature of patient data, federated learning (FL) is often leveraged to build models for smart healthcare applications. Existing deployed FL frameworks cannot address the key issues of varying data quality and heterogeneous data distributions across multiple institutions in this sector. In this paper, we report our experience developing and deploying the Contribution-Aware Federated Learning (CAreFL) framework for smart healthcare. It provides an efficient and accurate approach to fairly evaluate FL participants’ contributions to model performance without exposing their private data, and improves the FL model training protocol by allowing the best performing intermediate sub-models to be distributed to participants for FL training. Since its deployment by Yidu Cloud Technology Inc. in March 2021, CAreFL has served eight well-established medical institutions in China to build healthcare decision support models. It can perform contribution evaluations close to three times faster than the best existing approach and has improved the average accuracy of the resulting models by more than 2% compared to the previous system (which is significant in industrial settings). To the best of our knowledge, it is the first CAreFL successfully deployed in the healthcare industry.Nanyang Technological UniversityNational Research Foundation (NRF)Published versionThis research is supported by the National Research Foundation, Singapore and DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG2-RP-2020-019); Joint NTU-WeBank Research Centre on Fintech (Award No: NWJ-2020-008); the Nanyang Assistant Professorship (NAP); and the RIE 2020 Advanced Manufacturing and Engineering (AME) Programmatic Fund (No. A20G8b0102), Singapore. Qiang Yang is supported by the Hong Kong RGC theme-based research scheme (T41-603/20-R) and the National Key Research and Development Program of China under Grant No. 2018AAA0101100

Experimental and Modeling of Residual Deformation of Soil–Rock Mixture under Freeze–Thaw Cycles

Projects in seasonal frozen soil areas are often faced with frost heaving and thawing subsidence failure, and the foundation fill of most projects is a mixture of soil and rock. Therefore, taking soil–rock mixture with different rock contents as research objects, the residual deformation of soil–rock mixture under multiple freezing–thawing cycles is studied. In addition, the deep learning method based on the artificial neural network was pioneered combined with the freezing–thawing test of the soil–rock mixture, and the Long short-term memory (LSTM) model was established to predict the results of the freezing–thawing test. The LSTM model has been verified to be feasible in the exploration of the freeze–thaw cycle law of a soil–rock mixture, which can not only greatly reduce the period of the freeze–thaw test, but also maintain a high prediction accuracy to a certain extent. The study found that the soil–rock mixture will repeatedly produce frost heave and thaw subsidence under the action of freeze–thaw cycles, and the initial frost heave and thaw subsidence changes hugely. With the increase of the number of freeze–thaw cycles, the residual deformation decreases and then becomes steady. Under the condition that the content of block rock in the soil–rock mixture is not more than 80%, with the increase of block rock content, the residual deformation caused by the freeze–thaw cycle will gradually decrease due to the skeleton function of block rock, while the block rock content’s further increase will increase the residual deformation. Furthermore, the LSTM model based on an artificial neural network can effectively predict the freezing and thawing changes of soil–rock mixture in the short term, which can greatly shorten the time required for the freezing and thawing test and improve the efficiency of the freezing and thawing test to a certain extent

Contribution-Aware Federated Learning for Smart Healthcare

Artificial intelligence (AI) is a promising technology to transform the healthcare industry. Due to the highly sensitive nature of patient data, federated learning (FL) is often leveraged to build models for smart healthcare applications. Existing deployed FL frameworks cannot address the key issues of varying data quality and heterogeneous data distributions across multiple institutions in this sector. In this paper, we report our experience developing and deploying the Contribution-Aware Federated Learning (CAFL) framework for smart healthcare. It provides an efficient and accurate approach to fairly evaluate FL participants' contribution to model performance without exposing their private data, and improves the FL model training protocol to allow the best performing intermediate models to be distributed to participants for FL training. Since its deployment in Yidu Cloud Technology Inc. in March 2021, CAFL has served 8 well-established medical institutions in China to build healthcare decision support models. It can perform contribution evaluations 2.84 times faster than the best existing approach, and has improved the average accuracy of the resulting models by 2.62% compared to the previous system (which is significant in industrial settings). To our knowledge, it is the first contribution-aware federated learning successfully deployed in the healthcare industry

Efficient Training of Large-Scale Industrial Fault Diagnostic Models through Federated Opportunistic Block Dropout

Artificial intelligence (AI)-empowered industrial fault diagnostics is important in ensuring the safe operation of industrial applications. Since complex industrial systems often involve multiple industrial plants (possibly belonging to different companies or subsidiaries) with sensitive data collected and stored in a distributed manner, collaborative fault diagnostic model training often needs to leverage federated learning (FL). As the scale of the industrial fault diagnostic models are often large and communication channels in such systems are often not exclusively used for FL model training, existing deployed FL model training frameworks cannot train such models efficiently across multiple institutions. In this paper, we report our experience developing and deploying the Federated Opportunistic Block Dropout (FedOBD) approach for industrial fault diagnostic model training. By decomposing large-scale models into semantic blocks and enabling FL participants to opportunistically upload selected important blocks in a quantized manner, it significantly reduces the communication overhead while maintaining model performance. Since its deployment in ENN Group in February 2022, FedOBD has served two coal chemical plants across two cities in China to build industrial fault prediction models. It helped the company reduce the training communication overhead by over 70% compared to its previous AI Engine, while maintaining model performance at over 85% test F1 score. To our knowledge, it is the first successfully deployed dropout-based FL approach